Chapter 4 . Representation of Discrete

Recurrent neural networks are appropriate tools for modeling time-varying systems (e.g. nancial markets, physical dynamical systems, speech recognition, etc.). They can be used to recognize pattern sequences (e.g. speech recognition) or they can be used for forecasting future patterns (e.g. nancial markets). These applications are generally not well-suited for addressing fundamental issues of recurrent neural networks such as training algorithms and knowledge representation because they come with a host of application-speciic characteristics (e.g. nancial data is generally non-stationary, feature extraction may be necessary for speaker identiication, etc.) which may muddle the fundamental issues. This chapter introduces theoretical models of computation and formal languages as a convenient framework in which to study the computational capabilities of various network models.